Deformation field calculation apparatus, method, and computer readable storage medium

ABSTRACT

An image processing apparatus includes: a calculation unit configured to calculate deformation information of an object deformed from a first deformation state to a second deformation state; a degree-of-deviation calculation unit configured to calculate a degree of deviation of the deformation information with respect to a deformation model representing a deformation state of the object; and a display control unit configured to display the calculated degree of deviation.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image processing technique ofprocessing images captured by various image capturing apparatuses(modalities).

Description of the Related Art

In the medical field, image diagnosis is performed using medical images.A doctor finds a lesion portion (a portion which is suspected as alesion) based on imaging findings obtained from images, and determinesthe type, grade of malignancy, degree of progression, and the like of anabnormality. Japanese Patent Laid-Open No. 8-336517 (to be referred toas patent literature 1 hereinafter) discloses a technique of observing atemporal change of a lesion portion by aligning images of an objectcaptured at different dates/times with each other to display thedifference between the images.

However, image diagnosis requires more information representing thefeatures of a lesion portion in addition to information about hardnessobtained by elastography and/or a temporal change obtained by thetechnique described in patent literature 1. The present invention hasbeen made in consideration of the above problem, and provides newinformation representing the features of a lesion portion which isdeformed differently with respect to the behavior of the lesion portionwhen an object is deformed.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, there is provided animage processing apparatus comprising: a calculation unit configured tocalculate deformation information of an object deformed from a firstdeformation state to a second deformation state; a degree-of-deviationcalculation unit configured to calculate a degree of deviation of thedeformation information with respect to a deformation model representinga deformation state of the object; and a display control unit configuredto display the calculated degree of deviation.

According to another aspect of the present invention, there is providedan image processing apparatus comprising: an obtaining unit configuredto obtain an image by capturing an object in a first deformation stateand an image by capturing the object in a second deformation state; animage generation unit configured to generate, based on a modelrepresenting deformation of a normal tissue of the object, a deformedimage by deforming the image in the first deformation state into thesecond deformation state; a degree-of-deviation calculation unitconfigured to calculate a degree of deviation of the image in the seconddeformation state with respect to the deformed image; and a displaycontrol unit configured to display the calculated degree of deviation.

According to still another aspect of the present invention, there isprovided an image processing apparatus comprising: an obtaining unitconfigured to obtain an image by capturing an object in a firstdeformation state and an image by capturing the object in a seconddeformation state; a position obtaining unit configured to obtainposition information of a reference point indicating the same portion onthe object from each of the image captured in the first deformationstate and the image captured in the second deformation state; a settingunit configured to set a corresponding point of interest on the objectfor each of the image captured in the first deformation state and theimage captured in the second deformation state; a degree-of-deviationcalculation unit configured to calculate, as a degree of deviation, achange between a positional relationship between the reference point andthe point of interest in the image captured in the first deformationstate and a positional relationship between the reference point and thepoint of interest in the image captured in the second deformation state;and a display control unit configured to display the calculated degreeof deviation.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the functional arrangements of animage processing system and image processing apparatus according to thefirst embodiment;

FIG. 2 is a flowchart illustrating an overall processing procedureaccording to the first embodiment;

FIGS. 3A to 3D are views respectively exemplifying input images, a realdeformation field, and a normal deformation field according to the firstembodiment;

FIG. 4 is view showing an example of a degree-of-deviation displaymethod according to the first embodiment;

FIG. 5 is a block diagram showing the functional arrangements of animage processing system and image processing apparatus according to thesecond embodiment;

FIG. 6 is a flowchart illustrating an overall processing procedureaccording to the second embodiment;

FIG. 7 is a block diagram showing the functional arrangements of animage processing system and image processing apparatus according to thethird embodiment; and

FIG. 8 is a flowchart illustrating an overall processing procedureaccording to the third embodiment.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be exemplarily described indetail below with reference to the accompanying drawings. Note thatcomponents to be described in these embodiments are merely examples. Thetechnical scope of the present invention is defined by the scope of theclaims, and is not limited by the following embodiments.

First Embodiment: Deviation from Normal Deformation

An image processing apparatus according to this embodiment receives aplurality of three-dimensional tomographic images obtained by capturingthe same object in different deformation states, and visualizes anddisplays degrees of deviation from the deformation model of a normaltissue with respect to the deformation of the object between the images.The image processing apparatus according to this embodiment will bedescribed below.

<Arrangement of Image Processing System 1000>

FIG. 1 is a block diagram exemplifying the arrangement of the imageprocessing system 1000 according to this embodiment. The imageprocessing system 1000 includes an image processing apparatus 100, adata server 170, and a display unit 180. Note that the components of theimage processing system 1000 are not limited to these apparatuses, andthe image processing system may be configured to further include othercomponents or not include some of the components.

The data server 170 holds a plurality of three-dimensional tomographicimages (volume images) obtained by capturing in advance the same object(target case) in different deformation states (different body postures,different conditions of respiration, or the like). An image capturingapparatus (modality) for capturing a three-dimensional tomographic imageis, for example, an MRI apparatus, X-ray CT apparatus, three-dimensionalultrasonic imaging apparatus, photoacoustic tomography apparatus, a PET(Positron Emission Tomography)/SPECT (Single Photon Emission ComputedTomography), or OCT apparatus. In this embodiment, the images aredesirably captured by the same modality in the same imaging mode atalmost the same time. Note that a combination of images is not limitedto this. As long as sufficient deformation occurs between images, theimages may be obtained by capturing the object by different modalitiesor in different imaging modes. The images may be obtained by capturingthe same patient at different dates/times. A case in which the dataserver 170 targets breast MRI images and holds the first image obtainedby capturing an object in a supine position and the second imageobtained by capturing the object in a prone position will be exemplifiedbelow.

FIG. 3A is a view showing an example of the first image and FIG. 3B is aview showing an example of the second image. Note that the deformationstate of the object when capturing the first image will be referred toas the first deformation state hereinafter, and the deformation state ofthe object when capturing the second image will be referred to as thesecond deformation state hereinafter. A data obtaining unit 104 obtainsthe images of the object held in the data server 170, and inputs them tothe image processing apparatus 100.

The data server 170 further holds the deformation model of a normaltissue when the object is deformed from the first deformation state intothe second deformation state. The deformation model is constructed inadvance based on the first images (supine position MRI images) obtainedby capturing a plurality of normal objects (normal cases) in the supineposition (first deformation state) and the second images (prone positionMRI images) obtained by capturing the normal objects in the proneposition (second deformation state). The deformation model is formedfrom two elements: a standard object image and a standard deformationfield (standard deformation information) defined on the space of thestandard object image.

The standard object image is generated by obtaining the average shape ofthe plurality of first images, projecting (overlaying) the voxel values(for example, intensities) of the first images onto the average shape,and then obtaining the average of the voxel values (intensities). Theaverage shape is obtained by performing deformable registration for thefirst images of all the normal cases, and obtaining the average shape ofthe first images. After that, the voxel values (intensities) of thefirst images are projected onto the average shape, and the average ofthe voxel values (intensities) is obtained, thereby generating thestandard object image. That is, the standard object image is an averageimage obtained by averaging the shapes and voxel values (intensities) ofthe first images, and projecting (overlaying) the average voxel value(intensity) onto the average shape. The standard object image can be animage of an average object (a supine position MRI image of an averagebreast) in the first deformation state.

On the other hand, the standard deformation field (standard deformationinformation) is obtained by projecting, onto the average shape, adeformation field (deformation information) from the first image to thesecond image, which has been estimated by performing deformableregistration for the first and second images of the normal cases, andperforming statistical processing by principal component analysis afterprojection. At this time, principal component analysis may be performedfor the displacement amount of each point obtained by sampling thedeformation field at a predetermined interval, or performed for thecontrol amount of each control point after parametrically expressing thedeformation field by FFD (Free-Form Deformation) or the like.

In either processing, the standard deformation field formed by anaverage deformation field and a principal component deformation field isgenerated. In the following description, T_(ave)(p) represents theaverage deformation field, and T_(eigen) _(_) _(i)(p) (i indicates thenumber of the principal component) represents the principal componentdeformation field. Each of the deformation fields can be regarded as afunction having a three-dimensional vector representing deformation fromthe first deformation state to the second deformation state with respectto each position p on the coordinate system of the standard objectimage. The thus constructed deformation model is a model (statisticdeformation model) representing statistically appropriate deformation(normal deformation) from the first deformation state to the seconddeformation state. That is, the deformation model is a modelrepresenting deformation from the first deformation state to the seconddeformation state and indicating how the normal object (normal case) isdeformed. Note that the standard object image and the standarddeformation field may be the first image of a typical normal case and adeformation field from the first image to the second image in thetypical normal case. The deformation model held in the data server 170is input to the image processing apparatus 100 via a deformation modelobtaining unit 102.

The display unit 180 displays display images generated by the imageprocessing apparatus 100 and various kinds of information such asdegrees of deviation. In addition, a GUI for obtaining an instructionfrom the user is arranged on the display unit 180.

<Functional Block Arrangement of Image Processing Apparatus 100>

The image processing apparatus 100 includes the deformation modelobtaining unit 102, the data obtaining unit 104, a normal deformationcalculation unit 106, a real deformation calculation unit 108, adegree-of-deviation calculation unit 110, and a display control unit112. A CPU (not shown) controls the overall processing of the imageprocessing apparatus 100, and controls the operation of each functionalblock by reading out a program and executing it.

The deformation model obtaining unit 102 obtains the deformation modelinput from the data server 170 to the image processing apparatus 100.The data obtaining unit 104 obtains the first and second images inputfrom the data server 170 to the image processing apparatus 100. Thenormal deformation calculation unit 106 calculates, in the form of adeformation field (to be referred to as a normal deformation fieldhereinafter) from the first image to the second image, deformation onthe assumption that the deformation of a target case complies with thedeformation model of a normal tissue, and obtains it. The realdeformation calculation unit 108 calculates deformation actuallyoccurring between the first and second images of the target case in theform of a deformation field (to be referred to as a real deformationfield hereinafter) from the first image to the second image, and obtainsit. The degree-of-deviation calculation unit 110 obtains the normaldeformation field from the normal deformation calculation unit 106, andobtains the real deformation field from the real deformation calculationunit 108. The degree-of-deviation calculation unit 110 calculatesdegrees of deviation based on information about the obtained normaldeformation field and real deformation field. The display control unit112 performs display control to display cross-sectional images of thefirst and second images and information about the degrees of deviationon the display unit 180.

<Processing Executed by Image Processing Apparatus 100>

A processing procedure executed by the image processing apparatus 100according to this embodiment will be described with reference to aflowchart shown in FIG. 2.

(Obtaining of Data: S200)

In step S200, the data obtaining unit 104 obtains the first and secondimages of the target case from the data server 170. The data obtainingunit 104 outputs the obtained images to the normal deformationcalculation unit 106, real deformation calculation unit 108, and displaycontrol unit 112.

(Obtaining of Deformation Model: S201)

In step S201, the deformation model obtaining unit 102 obtains thedeformation model from the data server 170. The deformation modelobtaining unit 102 outputs the obtained deformation model to the normaldeformation calculation unit 106.

(Calculation of Real Deformation Field: S202)

In step S202, the real deformation calculation unit 108 performsdeformable registration of the first and second images of the targetcase, and calculates the real deformation field (the deformation fieldof deformation which has actually occurred) from the first image to thesecond image. To calculate the real deformation field, the realdeformation calculation unit 108 can use a known registration methodsuch as an FFD (Free-Form Deformation) method, or an LDDMM (LargeDeformation Diffeomorphic Metric Mapping) method based on the imagesimilarity or corresponding feature points between the first image andthe second image. The real deformation calculation unit 108 outputs thecalculated real deformation field to the normal deformation calculationunit 106 and degree-of-deviation calculation unit 110. In the followingdescription, T_(real)(p) represents the real deformation field obtainedin this step where p represents an arbitrary coordinate point on thefirst image, and T_(real)(p) represents actual displacement (realdeformation field) at the coordinate point p. FIG. 3C is a view showingan example of a real deformation field 303 obtained by the realdeformation calculation unit 108. FIG. 3C shows the distribution ofdeformation states actually occurring between the first and secondimages of the target case.

(Calculation of Normal Deformation Field: S203)

In step S203, the normal deformation calculation unit 106 calculates thenormal deformation field from the first image to the second image of thetarget case based on the deformation model obtained in step S201. Notethat the first and second images are images which have been obtained bythe normal deformation calculation unit 106 from the data obtaining unit104 in step S200. The normal deformation calculation unit 106 outputsthe calculated normal deformation field to the degree-of-deviationcalculation unit 110.

The normal deformation calculation unit 106 obtains the parameters ofthe deformation model best expressing the real deformation fieldT_(real)(p) obtained by the real deformation calculation unit 108 instep S202. The normal deformation calculation unit 106 obtains, as thenormal deformation field, a deformation field expressed by the obtainedparameters. More specifically, to obtain the normal deformation field,the normal deformation calculation unit 106 performs deformableregistration of the standard object image of the deformation model andthe first image of the target case, and obtains a deformation fieldT_(norm)(p) from the first image to the standard object image. Note thatto perform deformable registration between the images, the normaldeformation calculation unit 106 can use a known method, similarly tothe processing in step S202.

The normal deformation calculation unit 106 calculates coefficients oflinear combination α_(i) of T_(eigen) _(_) _(i)(p), which approximatesthe real deformation field T_(real)(p) most, by using the deformationfield T_(norm)(p) from the first image to the standard object image andthe standard deformation field (that is, the average deformation fieldT_(ave)(p) and principal component deformation field T_(eigen) _(_)_(i)(p)) of the deformation model. To do this, by using the coefficientsof linear combination α₁, . . . , α_(N) of the deformation model asunknown parameters, the “normal deformation” of the first image of thetarget case is given by:

T _(model)(p|α ₁, . . . ,α_(N))=T _(ave)(T _(norm)(p)+p)+Σ_(1≤i≤N)(α_(i)T _(eigen) _(_) _(i)(T _(norm)(p)+p))  (1)

where N represents the number of principal component deformation fields(principal component deformation fields T_(eigen) _(_) _(i)(p)) used asthe deformation model. A predetermined fixed value may be used as thevalue of N or the value of N may be changed, as needed, by the user ofthe image processing apparatus 100 via a GUI (not shown). The normaldeformation calculation unit 106 obtains the parameter α_(i) whichminimizes the residual (err) between the deformation field T_(model)(p)and the real deformation field T_(real)(p) at each point (each pointsampled at the predetermined interval) p within an object region Ω₁ ofthe first image by defining the residual by:

err=Σ_(pϵΩ1) |T _(real)(p)−T _(model)(p|α ₁, . . . ,α_(N))|  (2)

That is, the normal deformation calculation unit 106 estimates the“normal deformation” of the first image of the target case, whichapproximates the real deformation field T_(real)(p) most, based on theobtained parameter α_(i). The deformation field T_(model)(p) at thistime is obtained as a normal deformation field T_(ideal)(p). Note thatthe normal deformation calculation unit 106 can use a known linearsolution to calculate the parameter α_(i), and analytically derive thevalue of the parameter α_(i) as the solution of a linear equation. Notethat a method of driving the parameter α_(i) by the normal deformationcalculation unit 106 is not limited to the method of minimizing the sum(err) of errors defined by equation (2). The normal deformationcalculation unit 106 can obtain the parameter α_(i) by, for example,excluding the points p at each of which the error is large by using aknown outlier removal algorithm such as RANSAC (RANdom SAmple Consensus)or M-estimation, and minimizing the sum of the errors at the remainingpoints. FIG. 3D is a view showing an example of a normal deformationfield 304 obtained by the normal deformation calculation unit 106.

(Calculation of Degrees of Deviation: S204)

In step S204, for each voxel within the object region Ω₁ on the firstimage, the degree-of-deviation calculation unit 110 calculates thedegree of deviation of displacement at a position corresponding to thevoxel. The degree-of-deviation calculation unit 110 generates a volumeimage (to be referred to as a degree-of-deviation image hereinafter)having the degree of deviation in each voxel as a voxel value. Thedegree-of-deviation calculation unit 110 outputs the generateddegree-of-deviation image to the display control unit 112.

Note that the degree-of-deviation calculation unit 110 can calculate adegree of deviation K(pj) in each voxel (a three-dimensional position isrepresented by pj) by:

K(pj)=|T _(real)(pj)−T _(ideal)(pj)|  (3)

Equation (3) indicates calculation of the norm of the difference betweenreal displacement (real deformation field) at the point pj of interestand displacement (normal deformation field) at the same point (point pj)based on the statistically normal deformation model. Therefore, if thedifference between the real displacement (real deformation field) at thepoint pj and the displacement (normal deformation field) based on thestatistically normal deformation model is small, the value of the degreeof deviation K(pj) is small. Alternatively, if the difference is large,the value of the degree of deviation K(pj) is large.

Note that if the degree of deviation K(pj) is calculated as a degree ofdeviation between the real deformation field T_(real)(pj) and the normaldeformation field T_(ideal)(pj), another method may be used forcalculation. For example, the degree-of-deviation calculation unit 110can calculate the degree of deviation based on the angle differencebetween a vector representing the real deformation field and a vectorrepresenting the normal deformation field. In this case, for example, ifthe angle difference is large, the degree-of-deviation calculation unit110 determines a high degree of deviation. If the angle difference issmall, the degree-of-deviation calculation unit 110 determines a lowdegree of deviation. In addition, the degree-of-deviation calculationunit 110 can obtain the determinants (Jacobian) of Jacobian matrices ofthe real deformation field and normal deformation field at the positionpj, and calculate a degree of deviation based on the difference betweenthe determinants. In this case, if a difference in volume change in alocal region between the real deformation field and the normaldeformation field is large, the degree-of-deviation calculation unit 110determines a high degree of deviation. The degree-of-deviationcalculation unit 110 can also obtain the value of equation (3) or aJacobian value at each point near the position pj, and use itsintegrated value as a degree of deviation. That is, thedegree-of-deviation calculation unit 110 can calculate the degree ofdeviation of deformation information for the deformation modelrepresenting the deformation state of the object. Note that the abovemethod is merely an example, and the degree-of-deviation calculationunit 110 can calculate the degree of deviation by a method based oncomparison between the real deformation field (actual displacement) andthe normal deformation field (displacement based on the statisticallynormal deformation model).

Note that the processing of calculating a degree of deviation based onthe deformation field with reference to the space of the first image andgenerating a degree-of-deviation image has been explained but thedegree-of-deviation calculation unit 110 can generate adegree-of-deviation image in the space of the second image instead ofthe space of the first image. In this case, with respect to each voxel(a three-dimensional position is represented by qj) within an objectregion Ω₂ on the second image, the degree-of-deviation calculation unit110 identifies the coordinate point pj on the first image, which isdisplaced to this voxel by inverse-conversion of the real deformationfield T_(real). That is, the degree-of-deviation calculation unit 110obtains pj satisfying qj=T_(real)(pj). The degree-of-deviationcalculation unit 110 obtains the degree of deviation K(pj) by the sameprocessing as that indicated by equation (3), and executes processing ofobtaining this value as the voxel value at the coordinate point qj onthe degree-of-deviation image. Note that the display control unit 112 ofthe image processing apparatus 100 can display, on the display unit 180,an arrangement (for example, a GUI) for prompting the user to select oneof the spaces of the first and second images to be used as a referencefor generating a degree-of-deviation image. In this case, the user canselect, via the GUI, the first or second image as a reference forgeneration of a degree-of-deviation image. The degree-of-deviationcalculation unit 110 executes processing of generating adegree-of-deviation image based on the user selection.

The degree-of-deviation calculation unit 110 can total the degrees ofdeviation in the respective voxels within the object region Ω₁ whilegenerating a degree-of-deviation image (or without generating adegree-of-deviation image), thereby calculating a degree of deviationfor the entire image. For example, the degree-of-deviation calculationunit 110 can obtain the average or maximum value of the degrees ofdeviation, the number of voxels in each of which the degree of deviationexceeds a predetermined value (threshold), or the like as informationrepresenting the degree of deviation for the entire image. This canquantify a general trend of the degree of deviation of the target casefrom the normal cases (normal deformation). The degree-of-deviationcalculation unit 110 divides the object region Ω₁ into anatomicallymeaningful regions (for example, as for a breast, A region, B region, Cregion, C′ region, D region, E region, and the like). Thedegree-of-deviation calculation unit 110 can total the degrees ofdeviation for each region, and quantify the obtained degree ofdeviation. In addition, the degree-of-deviation calculation unit 110 canperform processing of, for example, selecting a region where the degreeof deviation is highest or a region where the degree of deviationexceeds a predetermined condition, and determine a region of interestbased on the degree of deviation.

(Visualization of Degrees of Deviation: S205)

In step S205, the display control unit 112 displays information aboutthe degrees of deviation on the display unit 180. For example, thedisplay control unit 112 can perform display control to overlay a colormap (to be referred to as a degree-of-deviation map hereinafter) as amap indicating the distribution of the degrees of deviation on thecross-sectional image of the grayscale first image. With respect todisplay of the degree-of-deviation map, the display control unit 112 canpredefine the correspondence between the value of a degree of deviationand a display color in a table, and perform display control to overlay adisplay color determined according to the table on a corresponding voxelof the first image. In this way, the display control unit 112 cancontrol display of the color map.

Note that display form of the distribution of the degrees of deviationis not limited to overlaying a color map representing the degrees ofdeviation. For example, the display control unit 112 can perform displaycontrol to overlay the isopleths of the degrees of deviation on thecross-sectional image of the first image. Furthermore, when the usermoves a mouse cursor on the cross-sectional image of the first image,the display control unit 112 obtains the degree of deviation at acoordinate point indicated by the cursor from the degree-of-deviationimage in synchronism with the movement of the cursor. The displaycontrol unit 112 can perform display control to display the value of theobtained degree of deviation as character information near the cursor orat a predetermined position within the screen of the display unit 180.In addition, the display control unit 112 can control display of thedisplay unit 180 to perform display in a display form in which, withrespect to each region where the degree of deviation is equal to orhigher than a predetermined value, information indicating the region isoverlaid on the cross-sectional image of the first image and nothing isoverlaid on the remaining regions. Note that the display form of thedistribution of the degrees of deviation is not limited to overlaying onthe first image, and the display control unit 112 can also controldisplay of the display unit 180 so as to display the degree-of-deviationimage of the same section as that of the first image beside the firstimage so that the degree-of-deviation image is readily compared with thefirst image. The display control unit 112 can perform display control ofthe display form of the distribution of the degrees of deviation inaccordance with a user instruction. FIG. 4 exemplifies, as a displayexample of the distribution of the degrees of deviation, a case in whicha degree-of-deviation map 401 defined on the first image is overlaid ona cross-sectional image 301 of the first image. For example, a lesionportion of a tomographic image obtained by an ultrasonic echo or thelike tends to be darkly captured, and the degrees of deviation areoverlaid on the captured image, thereby allowing a doctor to quicklyspecify the lesion portion in the image. With images captured by variousimage capturing apparatuses (modalities) and degree-of-deviationinformation, it becomes easy for a doctor to discover a lesion portionand determine the type, grade of malignancy, degree of progression, andthe like of an abnormality.

Note that if information representing the degree of deviation of theentire image, information representing the degree of deviation for eachanatomically meaningful region, information of region of interest basedon the degree of deviation, and the like have been derived in step S204,the display control unit 112 displays the pieces of information on thedisplay unit 180.

The display control unit 112 can extract the region of interest based onthe degree-of-deviation image, and perform display control to displayinformation of the region together with the cross-sectional image. Forexample, the display control unit 112 can extract a region where thedegree of deviation is equal to or higher than a predetermined valuefrom the degree-of-deviation image, determine a region satisfying apredetermined condition such as a volume as a region of interest, andperform display control to display information (number, volume,barycentric position, and the like) of the region.

As described above, the processing of the image processing apparatus 100is executed. The image processing apparatus 100 according to thisembodiment can confirm the presence/absence of deviation from the normaldeformation and a region where deviated deformation has occurred. Forexample, in diagnosis of an infiltrating cancer presenting disturbanceof tissue construction, even if it is difficult to detect aninfiltrating cancer by a touch or findings in an ultrasonic image, aregion where disturbance of tissue construction has occurred has afeature of impairing tissue elongation, and thus this can be visualizedas deviation from the normal deformation.

In addition, in diagnosis of a noninfiltrating cancer, a noninfiltratingcancer has a feature of pulling surrounding tissues inward to causedimpling, and thus this can be visualized and presented as deviationfrom the normal deformation. Since the image processing apparatusaccording to this embodiment can visualize and present thepresence/absence of deviation from the normal deformation and a regionwhere deviated deformation has occurred, it can support benign/malignantdiscrimination by a doctor in diagnosis of an infiltrating cancer anddiagnosis of a noninfiltrating cancer.

<Modification 1 of First Embodiment: Example of Deformation Model Exceptfor Statistic Deformation>

In the first embodiment, a case in which a model representingstatistically appropriate deformation is used as a deformation model byperforming statistical processing in deformation from the firstdeformation state to the second deformation state has been exemplified.However, another deformation model may be used. For example, as adeformation model representing physically appropriate deformation, afinite element model (physical model) for a physical deformationsimulation may be used. If, for example, the first image is used as areference, the normal deformation calculation unit 106 generates, as anormal deformation field, a deformation field obtained by performing aphysical deformation simulation for the first image. In generation of anormal deformation field, the normal deformation calculation unit 106generates a normal deformation field by obtaining deformationrepresenting a real deformation field by a physical deformationsimulation as much as possible under constraints based on the physicalmodel. This enables the degree-of-deviation calculation unit 110 toobtain the degree of deviation of the real deformation field withrespect to the physically appropriate deformation (the normaldeformation field based on the physical deformation simulation).

The normal deformation calculation unit 106 can use, as a deformationmodel representing anatomically appropriate deformation, a knowledgemodel obtained by modeling anatomical knowledge about the deformation ofthe object. In this case, the normal deformation calculation unit 106generates a normal deformation field by obtaining deformationrepresenting the real deformation field as much as possible inaccordance with the knowledge model. This enables thedegree-of-deviation calculation unit 110 to obtain the degree ofdeviation of the real deformation field with respect to deformationmatching the anatomical knowledge. Note that according to thismodification, the images of a plurality of normal cases for constructinga statistic deformation model become unnecessary.

<Modification 2 of First Embodiment: Example of Calculating RealDeformation Field by Interactive Operation>

In the first embodiment, a case in which the real deformation field iscalculated using image information (the first and second images) in theprocessing of the real deformation calculation unit 108 in step S202 hasbeen exemplified. However, generation of the real deformation field isnot limited to this processing. For example, pieces of positioninformation (corresponding point information) of corresponding pointsbetween the first and second images are manually or automatically input,and the real deformation calculation unit 108 can calculate a realdeformation field using the pieces of corresponding point information.In this case, in the processing in step S200, the data obtaining unit104 obtains the three-dimensional positions of points input with respectto the second image and the three-dimensional positions of points on thefirst image corresponding to the points on the second image. In theprocessing in step S202, the real deformation calculation unit 108 adds,as a constraint condition, displacement of the three-dimensionalpositions of the points on the first image to those of the correspondingpoints on the second image when calculating a real deformation field.According to this modification, even if it is impossible toappropriately obtain a real deformation field by performing onlyprocessing using image information as in the case in which the first andsecond images are obtained by different modalities, it is possible toappropriately calculate a real deformation filed by using thecorresponding point information.

<Modification 3 of First Embodiment: Normal Deformation FieldCalculation Method>

In the first embodiment, in the processing in step S203, the normaldeformation calculation unit 106 derives the normal deformation field byobtaining the parameter α_(i) so as to approximate the real deformationfield most. However, the normal deformation field may be derived byprocessing different from the above one. For example, the normaldeformation calculation unit 106 may derives α_(i) by obtaining, amongdeformation fields which can be represented by the deformation model ofequation (1), a deformation field in which the result of deforming thefirst image by the deformation field is closest to (matches) the secondimage, and sets the obtained deformation field as a normal deformationfield. At this time, the degree of matching between the images can beevaluated by, for example, the SSD (Sum of Squared Difference) of thevoxel values (intensities) at the same position. Furthermore, it ispossible to evaluate the degree of matching using a known evaluationcriterion such as the number of voxels in each of which the absolutevalue of the intensity difference is equal to or smaller than apredetermined value, the residual between the pieces of positioninformation of corresponding landmarks between the images, thesimilarity of the feature information between the images, or across-correlation.

Second Embodiment: Deriving of Shift from Preliminary Knowledge

In the first embodiment, the processing of obtaining a degree ofdeviation by comparing the real deformation field between the pluralityof three-dimensional tomographic images and the normal deformation fieldbased on the deformation model has been explained. On the other hand, animage processing apparatus according to this embodiment has as its mainfeature to calculate a degree of deviation based on preliminaryknowledge about the deformation of a normal tissue without calculating anormal deformation field based on a deformation model. Only thedifferences from the first embodiment in the image processing apparatusaccording to this embodiment will be described below.

Assume that an object according to this embodiment is a breast. As for anormal tissue, it is known that the distance (to be referred to as anipple-to-point distance hereinafter) between a nipple position and theposition of an arbitrary point in the breast is almost constantregardless of the deformation state. If there is a region where thenipple-to-point distance changes when the object is deformed from thefirst deformation state to the second deformation state, the region maydeviate from normal deformation. To cope with this, the image processingapparatus according to this embodiment obtains the degree of change ofthe nipple-to-point distance between the first image captured in thefirst deformation state and the second image captured in the seconddeformation state, and visualizes and displays the obtained degree ofchange of the nipple-to-point distance as a degree of deviation.

<Arrangements of Image Processing System 5000 and Image ProcessingApparatus 500>

FIG. 5 shows the arrangements of the image processing system 5000 andimage processing apparatus 500 according to this embodiment. Note thatthe same reference numerals as those in FIG. 1 denote the same parts anda description thereof will be omitted. The image processing apparatus500 includes a data obtaining unit 104, a real deformation calculationunit 108, a display control unit 112, a reference position obtainingunit 501, and a degree-of-deviation calculation unit 510.

The reference position obtaining unit 501 obtains pieces of positioninformation (in this embodiment, three-dimensional coordinate points ofnipples) of reference points in the first and second images as pieces ofinformation to be used to calculate a degree of deviation based onpreliminary knowledge. Based on the real deformation field calculated bythe real deformation calculation unit 108 and the pieces of positioninformation of the reference points obtained by the reference positionobtaining unit 501, the degree-of-deviation calculation unit 510calculates the degree of deviation of the real deformation field basedon the preliminary knowledge about normal deformation.

<Processing Executed by Image Processing Apparatus 500>

An overall processing procedure executed by the image processingapparatus 500 according to this embodiment will be described withreference to a flowchart shown in FIG. 6. Note that processes in stepsS600, S602, and S604 are the same as those in steps S200, S202, and S205of FIG. 2 described in the first embodiment and a description thereofwill be omitted.

(Obtaining of Position Information of Reference Point: S601)

In step S601, the reference position obtaining unit 501 obtains positioninformation (in this embodiment, the three-dimensional coordinate pointof a nipple position as an example of the position information of thereference point indicating the same portion of the object) of thereference point in the object to be used to calculate a degree ofdeviation from each of the first and second images. The obtained piecesof position information of the reference points (the three-dimensionalcoordinate points of the nipple positions) are output to thedegree-of-deviation calculation unit 510. The pieces of positioninformation (three-dimensional coordinate points of the nipplepositions) of the reference points in the first and second images arerepresented by n₁ and n₂, respectively.

(Calculation of Degrees of Deviation: S603)

In step S603, the degree-of-deviation calculation unit 510 generates adegree-of-deviation image having, as a voxel value, a degree ofdeviation in each voxel within an object region Ω₁ on the first image,and outputs the calculated degree-of-deviation image to the displaycontrol unit 112, as in the first embodiment. Note that in thisembodiment, a method of calculating a degree of deviation in each voxelis different from the first embodiment. That is, when each voxel is setas a point of interest, the degree-of-deviation calculation unit 510sets a corresponding point of interest on the object for each of thefirst and second images. The degree-of-deviation calculation unit 510calculates, as a degree of deviation, a change between the positionalrelationship between the reference point and the point of interest inthe first image and the positional relationship between the referencepoint and the point of interest in the second image.

As a change in positional relationship, the degree-of-deviationcalculation unit 510 calculates, as a degree of deviation, thedifference between the distance between the reference point and thepoint of interest (“the difference between the point of interest and thenipple (reference point)”) in the first image and the distance betweenthe reference point and the point of interest (“the difference between apoint corresponding to the point of interest and the nipple (referencepoint)”) in the second image. The difference between the distance valuesindicates a degree of deviation in this embodiment. Thedegree-of-deviation calculation unit 510 calculates a degree ofdeviation K(pj) at a voxel position pj by:

K(pj)=|pj−n ₁ |−|qj−n ₂|  (4)

where |pj−n₁| represents “the distance between the point of interest andthe nipple”, and qj represents the position of the point correspondingto the point of interest pj on the second image. That is, |qj−n₂|represents “the distance between the point corresponding to the point ofinterest and the nipple”. Calculation of equation (4) is equivalent tocalculation of the difference in the nipple-to-point distance at thesame point between the first and second deformation states. Note thatthe value of the position qj of the point corresponding to the point ofinterest pj on the second image is calculated using a real deformationfield obtained in step S602, as given by:

qj=pj+T _(real)(pj)  (5)

where T_(real)(pj) represents the real deformation field at thecoordinate point pj.

If the value of the degree of deviation K(pj) is close to 0, a change innipple-to-point distance at the point pj between the first and seconddeformation states is small. Based on knowledge that the nipple-to-pointdistance at the same point does not largely change in the normaldeformation state, it is considered that a region near the position pjhas been normally deformed. On the other hand, if the absolute value ofthe degree of deviation K(pj) is large, it is considered thatdeformation different from the normal model has occurred in a regionnear the position pj. Note that the degree-of-deviation calculation unit510 can obtain a degree of deviation by any method other than that usingthe difference indicated by equation (4) as long as the method canmeasure the difference in nipple-to-point distance.

As a change in positional relationship, the degree-of-deviationcalculation unit 510 calculates, as a degree of deviation, the ratiobetween the distance between the reference point and the point ofinterest in the first image and the difference between the referencepoint and the point of interest in the second image. For example, thedegree-of-deviation calculation unit 510 can calculate the ratio betweenthe nipple-to-point distances, that is, a degree of deviation by:

K(pj)=|pj−n ₁ |/|qj−n ₂|  (6)

The degree-of-deviation calculation unit 510 calculates a degree ofdeviation with reference to knowledge that the nipple-to-point distanceat the same point does not largely change in the normal deformationstate. Note that the degree-of-deviation calculation unit 510 maycalculate a degree of deviation by another method based on theinvariability of the positional relationship between the nipple and thepoint instead of the method based on the criterion that thenipple-to-point distance does not change.

The degree-of-deviation calculation unit 510 can use, as an example ofanother method based on the invariability of the positional relationshipbetween the nipple and the point, the criterion that the orientationbetween the nipple and the point with reference to the nipple does notlargely change due to deformation. The orientation is specificallydefined as follows. Assume that the reference coordinate system of anobject image is defined by setting the horizontal direction as theX-axis, the back-and-forth direction as the Y-axis, and caudal directionof the object as the Z-axis. Assume also that n={n_(x), n_(y), n_(z)}represents the nipple position, p={p_(x), p_(y), p_(z)} represents anarbitrary position in the object, and z={z_(x), z_(y), z_(z)} representsthe Z-axis vector (reference vector) on the reference coordinate system.In this case, the degree-of-deviation calculation unit 510 calculates anazimuth angle D(p) at the position p by:

D(p)=cos⁻¹{((p _(x) ,p _(z))−(n _(x) ,n _(z)))·(z _(y) ,z _(z))}   (7)

This calculation processing indicates calculation of an angle differenceby projecting the Z-axis vector (reference vector) and a vectorconnecting the position p (point of interest) and the nipple position n(reference point) onto the XZ plane on the reference coordinate system.The degree-of-deviation calculation unit 510 obtains azimuth anglesD(pj) and D(qj) at the position pj (point of interest) in the firstimage and at the corresponding position qj (point of interest) in thesecond image calculated by equation (5), and calculates the differencebetween the azimuth angles as a degree of deviation. That is, thedegree-of-deviation calculation unit 510 calculates, as a degree ofdeviation, the difference between angle information between thereference vector of the first image and a vector connecting thereference point and the point of interest in the first image and theangle information between the reference vector of the second image and avector connecting the reference point and the point of interest in thesecond image.

Note that the degree-of-deviation calculation unit 510 can use aplurality of criteria to calculate a degree of deviation instead ofusing a single criterion. If a degree of deviation is calculated bycombining a plurality of criteria, the degree-of-deviation calculationunit 510 weights degrees of deviation calculated based on the respectivecriteria by a weight parameter, and adds them, thereby calculating thefinal degree of deviation.

According to this embodiment, since clinically obtained anatomicalknowledge that the nipple-to-point distance does not change in thenormal deformation state can be directly used to calculate a degree ofdeviation, it is possible to calculate a degree of deviation moreadaptive to the intuition of the user such as a doctor. Furthermore,since it is unnecessary to derive a normal deformation field which isrequired in the first embodiment, it is possible to reduce thecalculation cost. In addition, since a statistic deformation modelrequired in the first embodiment is unnecessary, it is possible tovisualize deviation from the normal deformation without any input imageconstraints associated with postures such as “using a supine positionimage and a prone position image”.

Third Embodiment: Deriving of Deviation Based on Difference BetweenImages

In the first embodiment, a degree of deviation is calculated based onthe difference between the real deformation field between the pluralityof three-dimensional tomographic images and the normal deformation fieldbased on the deformation model. On the other hand, an image processingapparatus according to this embodiment has as its main feature tocalculate a degree of deviation based on the difference between a realimage and a deformed image generated by a normal deformation field basedon a deformation model without calculating a real deformation field.Only the differences from the first embodiment in the image processingapparatus according to this embodiment will be described below.

<Arrangements of Image Processing System 7000 and Image ProcessingApparatus 700>

FIG. 7 shows the arrangements of the image processing system 7000 andimage processing apparatus 700 according to this embodiment. Note thatthe same reference numerals as those in FIG. 1 denote the same parts anda description thereof will be omitted. The image processing apparatus700 includes a deformation model obtaining unit 102, a data obtainingunit 104, a normal deformation calculation unit 106, a display controlunit 112, a deformed image generation unit 707, and adegree-of-deviation calculation unit 710.

The deformed image generation unit 707 applies a normal deformationfield obtained by the normal deformation calculation unit 106 to thefirst image obtained by the data obtaining unit 104, thereby generatingan image (to be referred to as a deformed image or the third imagehereinafter) by deforming the first image into the second deformationstate based on the normal deformation field. The generated deformedimage (third image) is output to the degree-of-deviation calculationunit 710. The deformed image (third image) generated by the deformedimage generation unit 707 corresponds to an image deformed into thesecond deformation state by performing, for an object in the firstdeformation state, normal deformation which can be represented by anormal deformation model.

The degree-of-deviation calculation unit 710 calculates a degree ofdeviation based on the difference between the second image obtained bythe data obtaining unit 104 and the deformed image (third image)generated by the deformed image generation unit 707. The second imagerepresents the actual deformation state of the object in the seconddeformation state, and thus the difference between the real deformation(second image) and the normal deformation (third image) is quantified.

<Processing Executed by Image Processing Apparatus 700>

An overall processing procedure executed by the image processingapparatus 700 according to this embodiment will be described withreference to a flowchart shown in FIG. 8. Note that processes in stepsS800, S801, and S805 are the same as those in steps S200, S201, and S205of the first embodiment and a description thereof will be omitted.

(Calculation of Normal Deformation Field: S802)

In step S802, the normal deformation calculation unit 106 calculates anormal deformation field from the first image to the second image of atarget case based on a deformation model obtained in step S801. Thecalculated normal deformation field is output to the deformed imagegeneration unit 707. Note that since the image processing apparatus 700according to this embodiment has no processing unit corresponding to thereal deformation calculation unit 108 in the first embodiment, theprocessing in this step is executed by processing which is not based onthe real deformation field described in Modification 3 of the firstembodiment. The normal deformation calculation unit 106 derives α_(i) byobtaining, among deformation fields which can be represented by thedeformation model of equation (1), a deformation field in which theresult of deforming the first image by the deformation field is closestto (matches) the second image, and sets the obtained deformation fieldas a normal deformation field.

(Generation of Deformed Image: S803)

In step S803, the deformed image generation unit 707 generates adeformed image (third image) obtained by deforming the first image basedon the normal deformation field obtained in step S802. The deformedimage generation unit 707 outputs the generated deformed image (thirdimage) to the degree-of-deviation calculation unit 710. Generation ofthe deformed image based on the deformation field can be implemented bya known method.

(Calculation of Degrees of Deviation: S804)

In step S804, the degree-of-deviation calculation unit 710 calculates adegree of deviation of the real deformation occurring in each voxel fromthe normal model based on the deformed image (third image) obtained instep S803 and the second image obtained from the data obtaining unit104. The degree-of-deviation calculation unit 710 generates adegree-of-deviation image having the degree of deviation in each voxelas a voxel value. In this embodiment, the difference between the voxelvalues of corresponding voxels in the second image and the deformedimage (third image) is defined as a degree of deviation. As a practicalcalculation method, the degree-of-deviation calculation unit 710 cancalculate a degree of deviation by using, for example, the differencevalue between the voxel values of corresponding voxels or the absolutedifference values of the voxel values. Instead of calculating a degreeof deviation based on the direct difference between the second image andthe deformed image (third image), the degree-of-deviation calculationunit 710 can calculate a degree of deviation based on the differencebetween the images having undergone preprocessing such as smoothing. Byperforming preprocessing such as smoothing, it is possible to reduce theinfluence of noise or the like in degree-of-deviation calculationprocessing, thereby accurately obtaining a degree of deviation.

According to this embodiment, it is possible to obtain a degree ofdeviation from the normal deformation state without calculating the realdeformation field between the first and second images. Consequently, itis possible to reduce the calculation cost of the real deformationfield. Furthermore, even if it is difficult to calculate realdeformation due to a factor such as large deformation between the firstand second deformation states, it is possible to calculate a degree ofdeviation.

According to each of the above-described embodiments, it is possible toprovide new information representing the features of a lesion portionwhich is deformed differently with respect to the behavior of the lesionportion when an object is deformed.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2014-225433, filed Nov. 5, 2014, which is hereby incorporated byreference herein in its entirety.

1. An image processing apparatus comprising: at least one processor; andat least one memory storing instructions that, when executed by the atleast one processor, cause the at least one processor to function as: anobtaining unit configured to obtain a first image of a breast and asecond image of the breast different from the first image; a firstposition obtaining unit configured to obtain position information of anipple of the breast in each of the first image and the second image; asecond position obtaining unit configured to obtain position informationof a point of interest in the breast for each of the first image and thesecond image; a distance obtaining unit configured to obtain, on thebasis of the position information of the nipple and the positioninformation of the point of interest, information on a first distancebetween the nipple and the point of interest in the first image andinformation on a second distance between the nipple and the point ofinterest in the second image; a degree-of-deviation calculation unitconfigured to calculate, on the basis of the information on the firstdistance and the information on the second distance, adegree-of-deviation between the first distance and the second distance;and a display control unit configured to display information on thedegree-of-deviation on a display unit.
 2. The image processing apparatusaccording to claim 1, wherein the degree-of-deviation calculation unitis configured to calculate, as the degree-of-deviation, a differencebetween the first distance and the second distance.
 3. The imageprocessing apparatus according to claim 1, wherein thedegree-of-deviation calculation unit is configured to calculate, as thedegree-of-deviation, a ratio between the first distance and the seconddistance.
 4. An image processing apparatus comprising: at least oneprocessor; and at least one memory storing instructions that, whenexecuted by the at least one processor, cause the at least one processorto function as: an obtaining unit configured to obtain a first image ofa breast and a second image of the breast different from the firstimage; a first position obtaining unit configured to obtain positioninformation of a nipple of the breast in each of the first image and thesecond image; a second position obtaining unit configured to obtainposition information of a point of interest in the breast for each ofthe first image and the second image; an angle obtaining unit configuredto: obtain, on the basis of the position information of the nipple andthe position information of the point of interest, information on afirst angle between a reference vector of the first image and a vectorconnecting the nipple and the point of interest in the first image; andobtain, on the basis of the position information of the nipple and theposition information of the point of interest, information on a secondangle between a reference vector of the second image and a vectorconnecting the nipple and the point of interest in the second image; adegree-of-deviation calculation unit configured to calculate, on thebasis of the information on the first angle and the information on thesecond angle, a degree-of-deviation between the first angle and thesecond angle; and a display control unit configured to displayinformation on the degree-of-deviation on a display unit.
 5. The imageprocessing apparatus according to claim 4, wherein thedegree-of-deviation calculation unit is configured to calculate, as thedegree-of-deviation, a difference between the first angle and the secondangle.
 6. The image processing apparatus according to claim 4, whereinthe degree-of-deviation calculation unit is configured to calculate, asthe degree-of-deviation, a ratio between the first angle and the secondangle.
 7. An image processing method comprising: obtaining a first imageof a breast and a second image of the breast different from the firstimage; obtaining position information of a nipple of the breast in eachof the first image and the second image; obtaining position informationof a point of interest in the breast for each of the first image and thesecond image; obtaining, on the basis of the position information of thenipple and the position information of the point of interest,information on a first distance between the nipple and the point ofinterest in the first image and information on a second distance betweenthe nipple and the point of interest in the second image; calculating,on the basis of the information on the first distance and theinformation on the second distance, a degree-of-deviation between thefirst distance and the second distance; and displaying information onthe degree-of-deviation on a display unit.
 8. The image processingmethod according to claim 7, wherein a difference between the firstdistance and the second distance is calculated as thedegree-of-deviation.
 9. The image processing method according to claim7, wherein a ratio between the first distance and the second distance iscalculated as the degree-of-deviation.
 10. An image processing methodcomprising: obtaining a first image of a breast and a second image ofthe breast different from the first image; obtaining positioninformation of a nipple of the breast in each of the first image and thesecond image; obtaining position information of a point of interest inthe breast for each of the first image and the second image; obtaining,on the basis of the position information of the nipple and the positioninformation of the point of interest, information on a first anglebetween a reference vector of the first image and a vector connectingthe nipple and the point of interest in the first image; obtaining, onthe basis of the position information of the nipple and the positioninformation of the point of interest, information on a second anglebetween a reference vector of the second image and a vector connectingthe nipple and the point of interest in the second image; calculating,on the basis of the information on the first angle and the informationon the second angle, a degree-of-deviation between the first angle andthe second angle; and displaying information on the degree-of-deviationon a display unit.
 11. The image processing method according to claim10, wherein a difference between the first angle and the second angle iscalculated as the degree-of-deviation.
 12. The image processing methodaccording to claim 10, wherein a ratio between the first angle and thesecond angle is calculated as the degree-of-deviation.
 13. Anon-transitory computer-readable storage medium storing a program forcausing a computer to: obtain a first image of a breast and a secondimage of the breast different from the first image; obtain positioninformation of a nipple of the breast in each of the first image and thesecond image; obtain position information of a point of interest in thebreast for each of the first image and the second image; obtain, on thebasis of the position information of the nipple and the positioninformation of the point of interest, information on a first distancebetween the nipple and the point of interest in the first image andinformation on a second distance between the nipple and the point ofinterest in the second image; calculate, on the basis of the informationon the first distance and the information on the second distance, adegree-of-deviation between the first distance and the second distance;and display information on the degree-of-deviation on a display unit.14. A non-transitory computer-readable storage medium storing a programfor causing a computer to: obtain a first image of a breast and a secondimage of the breast different from the first image; obtain positioninformation of a nipple of the breast in each of the first image and thesecond image; obtain position information of a point of interest in thebreast for each of the first image and the second image; obtain, on thebasis of the position information of the nipple and the positioninformation of the point of interest, information on a first anglebetween a reference vector of the first image and a vector connectingthe nipple and the point of interest in the first image; obtain, on thebasis of the position information of the nipple and the positioninformation of the point of interest, information on a second anglebetween a reference vector of the second image and a vector connectingthe nipple and the point of interest in the second image; calculate, onthe basis of the information on the first angle and the information onthe second angle, a degree-of-deviation between the first angle and thesecond angle; and display information on the degree-of-deviation on adisplay unit.